In addition to the interpolation process such as pixel interpolation, removal of noise included in a pixel signal is required. Noise can be removed using a plurality of pixels at a periphery of a position of a pixel to be processed. For example, noise is removed using a total of 4 pixels including two pixels at the periphery of the position of the pixel to be processed along the horizontal direction and two pixels at the periphery of the position of the pixel to be processed along the vertical direction or a total of 4 pixels in the diagonal directions of the position of the pixel to be processed. However, when a noise removal process is executed on the image data in which the pixels are added along the horizontal direction or along the vertical direction by simply using the pixels at the periphery of the position of the pixel to be interpolated, noise cannot be accurately removed. This is because the resolution differs between the horizontal direction and the vertical direction, and consequently, a degree of influence or a degree of correlation with respect to the position of the pixel to be processed differs between these directions. In addition, with the noise removal, the degree of degradation of resolution changes between the horizontal direction and the vertical direction, possibly resulting in an unnatural image.